Papers with American Sign Language
WLASL-LEX: a Dataset for Recognising Phonological Properties in American Sign Language (2022.acl-short)
Copied to clipboard
| Challenge: | Signed Language Processing (SLP) is a major form of NLP, but has been overlooked by the NLP community. |
| Approach: | They leverage existing resources to construct a large-scale dataset of American Sign Language signs annotated with six different phonological properties. |
| Outcome: | The proposed model outperforms existing approaches on signs unobserved during training. |
Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting (2023.findings-eacl)
Copied to clipboard
| Challenge: | Yin et al. ( 2021) calls for including sign language processing (SLP) in natural language processing research. |
| Approach: | They propose to use a sign language writing system to parse, factorize, decode and evaluate signed languages. |
| Outcome: | The proposed method achieves over 30 BLEU in a bilingual setup and over 20 BLUE in two multilingual setups. |
Improving Sign Recognition with Phonology (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing work does not consider sign language phonology, but none leverages it . a recent study has shown that sign language recognition models lack structure . |
| Approach: | They explicitly recognize the role of phonology in sign production to train models for isolated sign language recognition . they train models that take in pose estimations of a signer producing a single sign to predict its phonological characteristics . |
| Outcome: | The proposed model improves sign recognition accuracy by 9% on the WLASL benchmark . the study could accelerate linguistic research in the domain of signed languages . |
Linguistically-driven Framework for Computationally Efficient and Scalable Sign Recognition (L18-1)
Copied to clipboard
| Challenge: | a new general framework for sign recognition from monocular video is presented . the framework exploits state-of-the-art learning methods while incorporating features based on what we know about the linguistic composition of lexical signs. |
| Approach: | They propose a general framework for sign recognition from monocular video . they exploit state-of-the-art learning methods while incorporating features from linguistic information . |
| Outcome: | The proposed framework exploits state-of-the-art learning methods while incorporating features based on what we know about linguistic composition of lexical signs. |
Signbank: Software to Support Web Based Dictionaries of Sign Language (L18-1)
Copied to clipboard
Steve Cassidy, Onno Crasborn, Henri Nieminen, Wessel Stoop, Micha Hulsbosch, Susan Even, Erwin Komen, Trevor Johnston
| Challenge: | Auslan Signbank is an on-line dictionary for Australian Sign Language (Auslan) it was originally built to support the Auslan signbank web dictionary, but was re-implemented using Microsoft SQL Server. |
| Approach: | This paper describes the overall architecture of the Auslan Signbank system and its representation of lexical entries and associated entities. |
| Outcome: | The current version of Auslan Signbank is an open-source re-implementation of the original website, with features added to allow updates to the database by researchers. |
Neural Machine Translation Methods for Translating Text to Sign Language Glosses (2023.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art techniques common to low resource Machine Translation (MT) are applied to improve MT of spoken language text to Sign Language glosses. |
| Approach: | They propose to use data augmentation, semi-supervised Neural Machine Translation, transfer learning and multilingual NMT to improve MT of spoken language to Sign Language glosses. |
| Outcome: | The proposed models outperform previous work on two German SL corpora and are confirmed by human evaluation. |
Bidirectional Skeleton-Based Isolated Sign Recognition using Graph Convolutional Networks (2022.lrec-1)
Copied to clipboard
| Challenge: | a new method for computer-based sign recognition from video is proposed . it involves explicit detection of the start and end frames of isolated signs . |
| Approach: | They propose a skeleton-based method that involves explicit detection of start and end frames of signs . they apply a modified WLASL dataset with corrections to the gloss labeling . |
| Outcome: | The proposed method outperforms state-of-the-art methods on the modified WLASL dataset . it has a success rate of 77.43% and 94.54% for top-5 . |
ASL STEM Wiki: Dataset and Benchmark for Interpreting STEM Articles (2024.emnlp-main)
Copied to clipboard
Kayo Yin, Chinmay Singh, Fyodor Minakov, Vanessa Milan, Hal Daumé Iii, Cyril Zhang, Alex Lu, Danielle Bragg
| Challenge: | Deaf and hard-of-hearing students face significant barriers in accessing STEM education due to the scarcity of STEM resources in signed languages. |
| Approach: | They develop models to identify fingerspelled words in American Sign Language (ASL) given an English sentence and a video, the model detects which English phrase is fingerspelled in the clip. |
| Outcome: | ASL STEM Wiki is the first continuous signing dataset focused on STEM . it detects fingerspelled words and queries them for appropriate signs to suggest to interpreters. |
Unsupervised Discrete Representations of American Sign Language (2024.emnlp-main)
Copied to clipboard
| Challenge: | Modern NLP models use discrete tokens to represent continuous signals, such as videos, audio, or gestures . modalities that are continuous are difficult to use with discrete models, such a LLM . |
| Approach: | They propose a method that discretizes sequences of fingerspelling signs into tokens . they also propose 'loss function' to improve interpretability of the tokens. |
| Outcome: | The proposed method improves the performance of the tokenizer on downstream tasks. |
Improving Handshape Representations for Sign Language Processing: A Graph Neural Network Approach (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for sign language recognition process a signing sequence holistically, leaving handshape information implicit, which limits both recognition accuracy and linguistic analysis. |
| Approach: | They propose a graph neural network that separates temporal dynamics from static handshape configurations in continuous signing sequences. |
| Outcome: | The proposed approach achieves 46% accuracy across 37 handshape classes, compared to 25% for baseline methods. |